Zero-Shot Face-Based Voice Conversion: Bottleneck-Free Speech Disentanglement in the Real-World Scenario

Often a face has voice. Appearance sometimes strong relationship with one's In this work, we study how can be converted to voice, which is face-based voice conversion. Since there no clean dataset that contains and speech, conversion faces difficult learning low-quality problems caused by background noise or echo. Too much redundant information for face-to-voice also causes synthesis of general style speech. Furthermore, previous work tried disentangle speech bottleneck adjustment. However, it hard decide on the size bottleneck. Therefore, propose bottleneck-free strategy disentanglement. To avoid synthesizing utilize framewise facial embedding. It applied adversarial multi-scale discriminator model achieve better quality. addition, self-attention module added focus content-related features in-the-wild data. Quantitative experiments show our method outperforms work.